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ABSTRACT 



In a system including a collection of cooperating cache 
servers, such as proxy cache servers, a request can be 
forwarded to a cooperating cache server if the requested 
object cannot be found locally. An overload condition is 
detected if for example, due to reference skew, some objects 
are in high demand by all the clients and the cache servers 
that contain those hot objects become overloaded due to 
forwarded requests. In response, the load is balanced by 
shifting some or all of the forwarded requests from an 
overloaded cache server lo a less loaded one. Both central- 
ized and distributed load balancing environments are 
described. 
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LOAD BALANCING COOPERATING CACHE imbalance problem resulting from too many requests for hot 

SERVERS BY SHIFTING FORWARDED objects being simultaneously forwarded to the same proxy 

REQUEST server. The present invention addresses such a need. 

Cooperative caching, or remote caching, has been used in 

FIELD OF THE INVENTION 5 distributed file systems to improve system performance (see 

™ . "Cooperative caching: Using Remote Client Memory to 

Tlie present invention is related to load balancing among i^p,^^, p^j, ^^^^^^ Performance," by M. D. DahUn et al.. 

cooperating cache servers and in particular to load balancmg p^^. of 1st Symp. on Operating Systems Design and 

based on load conditions and a frequency that requests are Implementation, pp. 1-14, 1994). Here, the file caches of a 

forwarded from cooperating cache servers. collection of workstations distributed on a LAN are coor- 

BACKGROUND dinated to form a more effective overall file cache. Each 

workstation caches not only objects referenced by local 

The growth in the usage of the World Wide Web has been requests but also objects that may be referenced by requests 

increasing exponentially. As a result, response times for firom a remote workstation. Upon a local cache miss, a local 

accessing web objects can become imsatisfactorily slow, request can be sent to other client workstations where a copy 

One approach to improving web access time is to employ can be obtained, if found. Otherwise, the object is obtained 

one or more proxy cache servers between browsers and the from the object server. The emphasis here is mainly how to 

originating web servers. Examples of proxy cache servers maintain cache coherency in the face of updates and how to 

include a cluster of PC servers running Microsoft's Win- maintain cache hit ratios by moving a locally replaced object 

dows NTtm, such as the NETFINITY™ servers from IBM; 20 cache memory of another workstation. There is no 

and workstation servers running IBM's AIXtm operating dynamic load balancing. 

system, such as the IBM RS/6000™ or SP/2xm. In fact, more Cooperative caching is also used in collective proxy cache 

and more organizations, such as Internet Service Providers servers to reduce the access time. Upon a cache miss, instead 

(ISPS) and corporations, are using a collection of cooper- of going directly to the originating web server potentially 

ating proxy cache servers to help improve response time as 25 through a WAN, a cache server may forward the request to 

well as reduce traffic to the Internet. A collection of coop- obtain the object from a cooperating cache server in a LAN 

era ting cache servers have distinct advantages over a single or a regional area network. For example, upon a local cache 

cache server in terms of reliability and performance. If one miss in the SQUID system, a cache server multicasts a 

fails, requests can still be serviced by other cooperating request (using the Internet Cache Protocol (ICP)) to a set of 

cache servers. Requests can be distributed among the 3^ other cache servers (see "Squid Internet Object Cache", by 

servers, thus increasing scalability. Finally, the aggregate D. Wessels et al., http://squid.nlanr.net/). If their caches 

cache size is much larger so that it is more likely that a contain the requested object, these cooperating cache servers 

requested object will be found in one of the cache servers. reply with a message indicating such. The requested object 

With cooperating cache servers, a request that cannot be is then obtained from the cooperating cache server which 

serviced locally due to a cache miss can be forwarded to 35 responded first to the request, instead of from the original 

another cache server storing the requested object. As a web server on the Internet. However, if none rephes after a 

result, there are two kinds of requests that can come to a time-out period, then the requested object will be fetched 

cache server: direct requests and forwarded requests. Direct from the originating web server. Load imbalances can occur 

requests are those that are received directly from clients. at a cache server due to forwarded requests. 

Forwarded requests are those that come from other cooper- 40 Instead of multicasting, the CRISP system uses a logical 

ating cache servers on behalf of their clients due to cache central directory to locate an object cached on another proxy 

misses on the cache servers. With requests forwarded among server (see "Directory Structures for Scaleable Internet 

the cache servers, a cache server can easily become over- Caches" ,S. Gadde et al.. Technical Report CS- 1997-18, 

loaded if it happens to contain in-demand (or "hot") objects Dept. of Computer Science, Duke University, 1997). Here, 

that most clients are currently interested in, creating uneven 45 upon a cache miss, a cache server asks the directory server 

workloads among the cache servers. Uneven workloads can for the object. With central knowledge of the caches object 

create a performance bottleneck, as many of the cache storage, the directory server sends such a request to the 

servers are waiting for the same overloaded cache server to server whose cache includes the object. If found, the object 

respond to requests forwarded to it. Therefore, there is a is then sent to the requesting server while the original server 

need for a way to perform dynamic load balancing among a 50 continues to cache the object. If no cache has a copy of the 

collection of proxy cache servers. The present invention requested object, the requesting server obtains the object 

addresses such a need. from the originating web server through the Internet 

Load balancing is traditionally done by a front-end sched- (potentially through a WAN). Again, this can create a load 

uler which "evenly distributes" incoming direct requests imbalance at the cache server due to subsequent requests 

among the cache servers. For example, load balancing can 55 forwarded to this cache server. 

be done at the DNS level by manipulating a mapping table. Yet another way to locate an object on a cooperating cache 

such as is done by the NETRAtx proxy cache by Sun server is through a hash function. An example is the Cache 

Microsystems ("Proxy Cache Server, Product Overview", Array Routing Protocol (CARP) (see V, Valloppillil and K. 

white paper. Sun Microsystems, http://www.sun.com/), W. Ross, "Cache Array Routing Protocol vl.O," Internet 

Load balancing among a cluster of servers can also be done 60 Draft, http://ircache.nlanr.net/Cache/ICP/draft-vinod-carp- 

with a front-end router, such as the NETDISPATCHER™ vl-03.txt, February 1998). In CARP, the entire object space 

offered by IBM (see e.g., G. Goldszmidt and 0. Hunt, is partitioned among the cooperating cache servers, with one 

"NetDispatcher: A TCP Connection Router," IBM Research partition for each cache server. When a request is received 

Report, RC 20853, May 1997). Here, incoming requests are by a cache server from a configured client browser, a hash 

distributed by the NETDISPATCHER™ to the least loaded 65 function is applied to a key from the request, such as the 

server in the cluster However, these traditional approaches URL or the destination IP address, to identify the partition, 

distribute only "direct requests" and do not address a load If the hash partition is the assigned to requesting cache 
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server, then the request is serviced locally. Otherwise, it is centralized and distributed environments and various 

forwarded to the proper cache server in the identified hybrids thereof. For example, a distributed load monitor can 

partition. be used for monitoring and maintaining a local load 

SQUID, CRISP and CARP use the caches of other proxy condition, the forwarding frequency and ownership infor- 
servers to reduce the possibility of having to go through the ^ mation for cached objects on each cooperating cache server, 

WAN for a missed object. They differ in the mechanism for The cooperating cache servers can periodically exchange 

locating a cooperating cache server whose cache may con- maintain one or more of: the load condition information; 

tain a copy of the requested object. Each cache server forwarding frequency; and the ownership information, 

services two kinds of requests: direct requests and forwarded For example, the cooperating cache servers can exchange 
requests. Direct requests are those made directly from the lO information by piggybacking one or more of: the load 

browsers connected to the proxy server. Forwarded requests condition information; the forwarding frequency; and the 

are those made by cooperating cache servers whose caches ownership information, with one or more of the forwarded 

do not have the requested objects. In any event, depending requests and responses. 

on the types of objects a proxy server caches at a given In another example, an overloaded cooperating cache 
moment, its CPU could be overloaded because it is busy 15 server can identify a less loaded cooperating cache server; 

serving both direct and forwarded requests. and communicate a shift request and a copy of the cached 

object to the less loaded cooperating cache server (which 

SUMMARY OF THE INVENTION then caches the object), so that subsequent requests for the 

In accordance with the aforementioned needs, the present object will not be forwarded. Alternatively, an overloaded 

invention is directed to a method and system for balancing cooperating cache server can communicate the shift request 

the load across a collection of cache servers that process to the less loaded cooperating cache server, which then 

both direct and forwarded requests by shifting some or all obtains a copy of the object from an originating object 

forwarded requests to a less loaded cache server. server, in response to the shift request. In yet another 

For example, in a system including a collection of coop- alternative, the owning cache server can multicast the shift 

erating proxy cache servers, a request can be forwarded to ^^^^^^^ message to one or more of the other cooperating 

another cooperating server if the requested object cannot be ^^^^ ^^^^ subsequent forward requests wHl be 

found locally. Instead of fetching the object from the origi- shifted. 

nating web server through the Internet, a cache server can In a fuUy distributed implementation of the present 
obtain a copy from a cooperating cache server in a local area invention, the cooperating cache servers can each include a 
network or an intranet. The average response time for access distributed load monitor for monitoring and locally main- 
to an object can be significantly improved by the cooperat- taining load conditions, and also can maintain the forward- 
ing cache server. However, due to reference skew, some ing frequency and ownership information in a local copy of 
objects can be in high demand by all the clients. As a result, a caching table or by means of a hashing function. The 
the proxy cache servers that contain those hot objects can cooperating cache servers can modify the ownership infor- 
become overloaded by forwarded requests coming from mationby means of the local copy ofthe caching table or the 
other proxy cache servers, creating a performance bottle- hash function. 

neck. According to the present invention, we propose a load The present invention includes still other features for 
balancing method for a collection of cooperating proxy modifying the ownership for the object to a shared owner- 
cache servers by shifting some or all of the forwarded ship between at least two of the cooperating cache servers 
requests from an overloaded cache server to a less loaded and forwarding subsequent object requests to one or more 
one. less loaded shared owners of the object. If a decrease in the 
An example of a cache server load balancing method in load condition for a shared object is detected, the shared 
accordance with the present invention includes the steps of: ownership can be merged, in response to the decrease in the 
receiving forwarded requests from a cooperating cache 45 load condition. 

server in response to a cache miss for an object on the In yet another example, the shifting of one or more of the 

cooperating cache server; and shifting one or more of the forwarded requests based on the load conditon an the 

forwarded requests for the object between cooperating cache forwarding frequency can be accomplished by communicat- 

servers based on a load condition and a forwarding fre- ing a copy of the object from the owning cache server to one 

quency for the object. or more of the cooperating cache servers, so that subsequent 

The present invention also includes features for periodi- requests will not be forwarded (as long as the object remains 

cally monitoring the load condition on and the forwarding in the recipient's cache). 

frequency to the owning cache server; and proactively An example of a centralized environment in accordance 

shifting one or more subsequent forwarded requests for the with the present invention includes: a centralized logical 
cached object from the owning cache server to one or more 55 load monitor for maintaining the forwarding frequency and 

of the cooperating cache servers, in response to the moni- the load condition for the cooperating cache servers. The 

toring. Alternatively, the shifting step further includes the load monitor can include a logical directory server for 

step of checking the load condition and forwarding maintaining a load table for monitoring the load on the cache 

frequency, in response to the receipt of a forwarded request, servers and a caching table (or hash function) for monitoring 
In one example, the load condition of the cooperating cache eo the forwarding frequency and locating objects. The directory 

server is a weighted sum of a count of said forwarded server receives requests for object locations in other cache 

requests, and a count of direct requests to said cooperating servers for a locaUy missed object and forwards requests for 

cache server. In another example, the cache information is locally missed objects. The directory server load balances 

maintained at: each object level; or a partition of objects requests among the cooperating cache servers by manipu- 

65 la ting the caching table based on the load and the forwarding 
The present invention also includes various implementa- . frequency for a given object, in response to the requests for 

tions for performing the load balancing, including both object locations. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

ITiese and other features, aspects, and advantages of the 
present invention will become better understood with refer- 
ence to the following description, appended claims, and ^ 
accompanying drawings wherein: 

FIG. la shows an example of a system in a block diagram 
form employing a collection of proxy cache servers, wherein 
a centralized load balancing logic according to the present 
invention can be applied; 

FIG. lb shows another example' of a system in a block 
diagram form employing a collection of proxy cache servers, 
where a distributed load balancing logic according to the 
present invention can be applied; 

FIGS. 2a-b show examples of data formats for two tables is 
that can be maintained by the load monitor depicted in FIGS. 
a~b; 

FIG. 3 shows an example of a logic flow for the load 
monitor in response to a request from a cache server because 
of a cache miss; and 

FIG, 4 shows an example of a logic flow for a cache server 
in response to a request for an object. 

DETAILED DESCRIPTION 

25 

Examples of the load balancing logic of the present 
invention will be described for both centralized and distrib- 
uted, architectures. FIG, 1 a shows an example of a block 
diagram of a system employing a collection of proxy cache 
servers, where a centralized load balancing logic proposed in 
this invention can be applied. As depicted, the system 
includes a collection of proxy cache servers 150. Although 
only a single level of cache server is depicted, there could be 
a hierarchy of cache servers 150. As is conventional, these 
proxy cache servers are connected with each other through 35 
a local area network (LAN) or a regional area network or 
intranet 140. Each cache server 150 is also connected to a 
wide area network (WAN) or the Internet 110. Through the 
WAN, these proxy cache servers can reach 115 the origi- 
nating web servers for objects that cannot be found locally 4Q 
on their own caches. 

According to the present invention a logical load monitor 
120 includes a load balancing logic 130 for monitoring the 
load conditions and forwarding frequency (FIG. 2a) of the 
cooperating cache servers 150 and provides load balancing 45 
for them. As will be described below, various load monitor 
120 features can: reside in one or more of the cache servers; 
be duplicated and distributed among the cache servers; or 
reside in another dedicated system such as a personal 
computer (PC) server or workstation. In a centralized system 50 
configuration, the load monitor 120 can perform a central 
directory function in directing forwarded requests 125 to the 
cache servers. One or more browsers 160 can be configured 
to connect to each cache server 150. Direct requests 155 are 
sent from the clients such as computers running conven- 55 
tional browsers 160 to the configured cache server 150. If 
the requested object can be found locally, then it is returned 
to the browser. Otherwise, the cache server 150 communi- 
cates a message to the load monitor 120. Various example 
implementations of the load monitor 120 will be described go 
in more detail below. If no load imbalance condition or trend 
exits, the load monitor 120 then forwards the request 125 to 
the cache server 150 that owns the requested object. The 
owning cache server then sends the requested object to the 
requesting cache server, e.g., via the LAN 140. 55 

If an actual load imbalance is identified, or predicted 
based on a loading trend, the load monitor 120 initiates a 
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shifting of forwarded requests from the overloaded cache 
server to one or more underloaded (or less loaded) servers. 
As will be described in more detail below, the shifting of 
ownership can be based on the load condition of the servers 
150 and the forwarding frequency, as well as other factors. 

FIGS. 2a~b shows examples of data formats of two tables 
maintained by the load monitor. As depicted, the tables 
include a load table 102, and a caching table 101. One 
skilled in the art will appreciate that a single table, or various 
other data structures could alternatively or equivalently be 
used. The load table 102 includes the load condition 1021 of 
each (A,B,C . . . 1022) cache server 150 so that overloaded 
and underloaded servers can be identified:. As is 
conventional, load conditions 1021 can be updated periodi- 
cally by probing each cache server. The load of a cache 
server can be a weighted sum of the number of forwarded 
requests and the number of direct requests. An overloaded 
cache server 150 can be identified by any conventional 
techniques, e.g., the load monitor can compute the mean 
load of all proxy cache servers in past intervals. Overloaded 
cache servers can be those with loads exceeding a threshold 
above the mean load. According to the present invention, 
load balancing takes into account the amount of overloading 
as well as the load due to the forwarding frequency 1011 of 
the cached objects. This way, the load monitor can decide 
whether or not to continue shifting some or all forwarded 
requests from an overloaded cache server C 10213 to an 
underloaded server A 10211. The caching table 1010 
includes the forwarding frequency 1011 and ownership 1012 
information of an object or a partition of objects. As will be 
discussed below, the ownership can be single as in A 10122, 
or shared 10121, 10123 among two or more cooperating 
cache servers. The forwarding frequency 1011 represents the 
number of times a request for an object has been forwarded 
through the load monitor. In addition to the forwarding 
frequency 1011, the caching table 101 can also maintain a 
timestamp 1013, indicating the most recent time a request 
for an object was forwarded. Further, the caching informa- 
tion for an object or a partition 1010 can include a forward- 
ing frequency over a given time period (count/time) for the 
object ID or partition ID 1010 through the load monitor 120. 
Object partitions 1010 can alternatively be based on a hash 
function on object identifiers, or can be based on the 
directory structures that objects are organized by on the web 
servers. In the case of a partition, any object belonging to a 
partition will be forwarded by the load monitor. The shifting 
of ownership can be based on the load condition of the 
servers, the forwarding frequency 1011 and other informa- 
tion such as the time stamp information. 

FIG. 3 shows an example of a logic flow for steps taken 
by the load monitor 120 in response to a request 125 from 
a cache server 150 because of a cache miss. As depicted, in 
step 201, it checks to see if the requested object/partition can 
be found in the caching table. If not, in step,202, a new entry 
is created for the object/partition and a cache server is 
assigned as its owner. After the entry is located in the 
caching table, in step 203, the forwarding frequency 1011 is 
updated, e.g., incremented by 1. The load monitor then 
examines the load table 102 to see if the owner is currently 
overloaded (and that the forwarding frequency 1011 is a 
significant contributor thereto), in step 204. If yes, in step 
205, the load monitor finds an underloaded (or less loaded) 
cache server and assign it as the new 10122 (or shared) 
owner 10122 of the requested object. The ownership infor- 
mation 1012 for the object in the caching table 101 is 
updated accordingly. Those skilled in the art will appreciate 
that the logic flow could comprise a shared 10123 or 
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hierarchical ownership 1012 in the caching table 101 or cache servers where each cache server 150 multicasts to a 

other data structure employed. The request (possibly with a list of cooperating cache servers to locate a copy of a locally 

copy of the requested object) can then be forwarded 125 to missed object. In this case, no specific ownership inform a- 

a new sole 10122 (or shared 10123) owner, in step 206. tion need be maintained anywhere in the system. However, 

Alternatively, the new owner can be requested to obtain 115 5 there is also no guarantee of finding an object from the 

an object copy from the originating object server, e.g., via cooperating cache servers, either. Assume that a logical load 

the Internet 110. Those skilled in the art will appreciate that monitor 120 is used to maintain the load conditions 1021 of 

the load checking step 204 can be performed proactively, all proxy cache servers and share this information with each 

i.e., periodically or in response to an identified overload or cache server 150. The load balancing can be achieved by 

overload trend 1021 due at least in part to a high forwarding excluding overloaded servers from the list of cooperating 

frequency 1011— for a given object id/partition id 1010 and servers to which a cache server multicasts its request (also 

cache server (ownership 1012). If so, then in step 205, the called a shift request). As a result, only less loaded cache 

load monitor finds an underloaded (or less loaded) cache servers will receive forwarded requests 125 

server assigns it as the new (or shared) owner of the ^^^^er alternative is a load balancing method for a 

requested object, and possibly sends a copy of the object to ^^^^^^^^^ cx)operating proxy cache servers where a hash 

the new (or shared) owner as above. Conversely, if a shared ^5 . ^ & J ^ l- 

ownership model is used, in step 208, when the load ^^nction is used to locate a copy of a 1^^^^^ 

condition 10211 and forwarding frequency 10111 for a In this case, the object space can be partitioned among the 

shared ownership object (p 10101) drops below a predeter- cooperating proxy cache servers 150, with one partition for 

mined threshold, in step 209, the shared ownership (B, A ^^^^ ^^^^^ ^^^^^ achieve load balancing by 

10121) can be merged to a single ownership and one of the 20 shining forwarded requests, one can change the hash func- 

copies purged from one of the cache servers A 10121, e.g., ^io" so that forwarded requests will not go to overloaded 

to make room for another hot object. servers. One preferred approach is to hash the object space 

FTG. 4 shows an example of a logic flow for a cache server into a large number of buckets, much larger than the total 

when a request for an object is received, either directly 155 number of proxy cache servers. These hash buckets are then 

from a browser 160 or forwarded 125 from the load monitor 25 assigned to the cache servers, with the goal of balancing the 

120. As depicted, in step 301, it first checks to see if the loads among them. Periodically, one can move one or more 

requested object can be found locally in its cache. If yes, in hash buckets from one overloaded server to an underloaded 

step 302, it returns the object and the process ends, in step server, effectively changing the hash function. 

306. Otherwise, in step 303, it checks to see if the request is In either case, the load condition of the cooperating cache 

a direct request or a forwarded request. If it is a direct 3Q server can factor in the forwarding frequency directly into 

request, in step 304, the request is sent to the load monitor the calculated load condition. For example, the load condi- 

and the process ends, in step 306. On the other hand, if the tion can be a weighted sum of a count of said forwarded 

request is a forwarded request, in step 305, the cache server requests, and a count of direct requests to said cooperating 

will fetch the object from the originating web server and cache server. Alternatively, the load monitor could sepa- 

retum the object. The process then ends, in step 306. 35 rately maintain the overal forwarding frequency for each 

Referring now to FIGS, la and 2a-b, assume for example, cooperating cache server, 

a browser 160 connecting to a cache server C 10223 requests Referring now to FIGS, lb and 2a-b, yet another alter- 

155 an object p 10101. From the caching table 101, it can be native is a load monitor 120 that is distributed, i.e., wherein 

seen that object p 10101 is not cached on server C, but it is some or all the load monitor is duphcated across the cache 

cached on ("owned" by) cache server B (assuming B, A 40 servers 150. In one example, the distributed load monitor 

10121 is initially only solely designated by B). In response includes local load condition information 1021 (and as 

to a cache miss on object p, server C 10223 sends a request described below, possibly the load conditions of all (A, B, 

to the load monitor 120 for object p. Depending on the load C, . . . 1022)) of the cooperating cache servers 150. The 

condition 10212 and forwarding frequency 1011 of requests distributed load monitor 120' preferably also includes the 

for p 10101 on server B, the load monitor may forward the 45 caching table 101 with the forwarding frequency 1011 and 

request to server B, asking it to send a copy of object p to ownership 1012 information for each object id/partition id 

server C. Or, if server B is currently overloaded or is 1010. Altematively, a hashing function, for example as 

trending as such, the load monitor might shift the forwarded described above, could be distributed and stored in the cache 

request by finding an underloaded (or less loaded) server to servers. Load condition information 1021 and/or caching 

serve as a new (or shared as in B, A 10121) owner of object 50 information 101: can be exchanged periodically; when there 

p. The request is then forwarded to the new (or shared e.g., is a change in status (ownership or significant change in load 

A) owning server for the object. Note that even after the condition); or piggybacked with cache forwarding requests 

transfer of ownership, a copy of object p is still on server B's and, responses. Load condition 1021 information could also 

cache and can still serve direct requests coming to server B. have a time stamp (not shown) associated with it for tracking 

However, in this example, all future forwarded requests for 55 or other pmposes. 

object p (or perhaps some, in the case of a shared ownership) Here, if a cache server 150 has a cache miss, the local load 

wiU be shifted to server A, Altematively, in the case of monitor 120' looks up the ownership of the requested object 

shared ownership B, A 10121, future forwarded requests for in its local caching table 101 and forwards the request, to the 

object p 10101 can be sent to the less loaded server. owning cache server. Altematively, the hash function could 

Now that a load balancing method according to the eo be applied to a key from the request, such as the URL or the 

present invention has been described for a collection of destination IP address, to identify the partition and the 

proxy cache servers where a logical central directory is used request then forwarded to the correct cache server. When the 

for locating an object, various alternatives will be consid- forwarded request (i.e., from a cache server who had a cache 

ercd. The present invention can be adapted to achieve load miss) is received, the owning cache server identifies it as a 

balancing for these systems as well. ^5 forwarded request (e.g., by identifying it as from another 

For example, the present invention can be configured to cache server as opposed to a client) and updates its forward- 
perform load balancing for a collection of cooperating proxy ing frequency 1011 information as applicable (FIG. 3, step 
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203) . If an overload trend or condition is indicated (step 

204) , the owning cache server can respond to the requesting 
cache server with a shift request and a copy of the cached 
object. Alternatively, the requesting cache server can obtain 

a copy from the originating object server via an intranet, 5 
WAN or Internet 110. In either case, when the forwarding 
server caches a copy of the object, this server will no longer 
issue forward requests (steps 301, 302) as long as it remains 
in the cache, thus proportionally reducing the load on the 
owning server. In addition, the owning cache server can 
multicast a shift request message to one or more of the other 
cooperating cache servers 150 so that subsequent forward 
requests will be shifted, e.g., by updating their local copy of 
the caching table or modifying the hash function (step 205). 
At this point, other cache servers can forward their requests ^5 
to the new owner (or to the least loaded owner of two^ or 
more cache servers 150 if ownership is shared) as indicated 
in their local copy of the caching table 101. When the 
original cache owner's load has decreased to an acceptable 
level (step 204), e.g., as indicated by a threshold, the shared 20 
ownership information can be merged to its original state 
(e.g., B,A 10121->B). 

In the case that the load condition information 1021 for all 
cache servers (A,B,C . . . 1022) is fully distributed, the 
requesting cache server could proactively check the load 25 
condition (and associated time stamp) of the owning server 
(step 204), i.e., before forwarding the request. If overloaded, 
the requesting server could request a copy of the object from 
the owning server (or from the originating server via the 
intranet or Internet 110) and possibly a load condition 30 
confirmation. The owning cache server could update its 
caching table 101 or modify the hash function to indicate the 
new shared ownership (step 205). The requesting server (or 
the owning server) could then multicast a message to all 
other cache servers 150 indicating the new shared ownership 35 
of the object and possibly include an updated load condition. 
At this point, other cache servers would update their caching 
tables 101 or modify the hash function to indicate the new 
shared ownership (step 202), and can forward their requests 
(step 206) to the least loaded owner of two or more cache 40 
servers 150 sharing ownership as indicated in their local 
copy of the caching table 101. When a shared cache owner's 
load has decreased to an acceptable level (steps 204 and 
208), e.g., as indicated by a threshold, the ownership infor- 
mation can be merged to its original state, in step 209. 45 

A preferred embodiment of the present invention includes 
features that can be implemented as software tangibly 
embodied on a computer program product or program 
storage device for execution on a processor (not shown) 
provided with cache server 150 or other computer embody- 50 
ing the load monitor 120, such as in the centralized model 
described. For example, software implemented in a popular 
object-oriented computer executable code such as JAVA 
provides portability across different platforms. Those skilled 
in the art will appreciate that many other compiled or 55 
interpreted, procedure -oriented and/or object-oriented (00) 
programming environments, including but not Umited to 
REXX, C, C"*" and Smalltalk can also be employed. 

Those skilled in the art will also appreciate that methods 
of the present invention may be the software may be 60 
embodied on a magnetic, electrical, optical, or other persis- 
tent program and/or data storage device, including but not 
limited to: magnetic disks. Direct Access Storage Devices 
(DASD), bubble memory; tape; optical disk formats such as 
CD-ROMs and DVD; and other persistent (also called 65 
nonvolatile) storage devices such as core, ROM, PROM, 
flash memory, or battery backed RAM. Those skilled in the 
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art will appreciate that within the spirit and scope of the 
present invention, one or more of the components instanti- 
ated in the memory of the server 120' could be accessed and 
maintained directly via disk (not shown), the network, 
another server, or could be distributed across a plurality of 
servers. 

While we have described our preferred embodiments of 
our invention with alternatives, it will be understood that 
those skilled in the art, both now and in the future, may make 
various improvements and enhancements which fall within 
the scope of the claims which follow. These claims should 
be construed to maintain the proper protection for the 
invention first disclosed. 

We claim: 

1. A cache server load balancing system, comprising: 
means for receiving forwarded requests which have been 

forwarded from one of a plurality of cooperating cache 
servers in response to a cache miss for an object on the 
cooperating cache server; and 
shifting means for shifting one or more of said forwarded 
requests for the object between cooperating cache 
servers based on dynamically maintained server load 
conditions and forwarding frequency, said forwarding 
frequency comprising the number of times requests for 
the object have been forwarded, 

2. The system of claim 1, said shifting means further 
comprising: 

means for periodically monitoring the load condition on 
and the forwarding frequency to said cooperating cache 
servers; and 

means for proactively shifting one or more subsequent 
forwarded requests for the cached object from between 
said cooperating cache servers, in response to said 
monitoring. 

3. The system of claim 1, said shifting means further 
comprising means for checking the load condition and 
forwarding frequency, in response to the forwarded request. 

4. The system of claim 1, wherein said shifting means 
comprises means for modifying an ownership for the object 
to a shared ownership between two or more of said coop- 
erating cache servers, 

5. The system of claim 4, further comprising merging said 
shared ownership in response to a change in the load 
condition. 

6. The system of claim 1, further comprising means for 
locally monitoring the load condition on each cooperating 
cache server. 

7. The system of claim 6, further comprising: 
distributed load monitor means for maintaining a local 

load condition, the forwarding frequency and owner- 
ship information for cached objects on said each coop- 
erating cache server. 

8. The system of claim 7, further comprising: . 

means for said cooperating cache servers to periodically 
exchange and maintain one or more of: the load con- 
dition information; the forwarding frequency; and the 
ownership information. 

9. The system of claim 7, further comprising the steps of: 
means for said cooperating cache servers to piggyback 

one or more of: the load condition information; the 
forwarding frequency; and the ownership information; 
with one or more of the forwarded requests and 
responses. 

10. The system of claim 7, further comprising: 

means for identifying a less loaded cooperating cache 
server; and 
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means for communicating one or more of: a shift request; 
and a copy of the cached object, to said less loaded 
cooperating cache server. 

11. The system of claim 10, further comprising: 

said less loaded cooperating cache server including means 5 
for receiving said shift request; and 

said less loaded cooperating cache server including means 
for requesting a copy of the object from an originating 
object server, in response to said shift request. 

12. The system of claim 10, wherein the copy is obtained 50 
via one or more of an intranet, WAN or Internet. 

13. The system of claim 7, further comprising: 

a local copy of a caching table on said each cooperating 
cache server; and 

means for said cooperating cache servers to maintain the 15 
forwarding frequency and the ownership information 
based on the local copy of the caching table. 

14. The system of claim 7, further comprising: 

hash function means for hashing the object space into a 
number of hash buckets much larger than a total 20 
number of said cache servers and assigning said hash 
buckets to said each cooperating cache server for 
locating a copy of a locally missed object; and 

said shifting means comprising means for moving one or 
more hash buckets from an overloaded server to a less ^5 
loaded server, effectively changing the hash function so 
that forwarded requests will not go to said overloaded 
server. 

15. The system of claim 14, further comprising: 
means for modifying the ownership for the object to a 

shared ownership between at least two of said cooper- 
ating cache servers; and 
means for said cooperating cache servers to forward 
subsequent object requests to one or more less loaded 
shared owners of the object. 

16. The system of claim 15, further comprising: 
means for detecting a decrease in the load condition for a 

shared object; and 
means for merging the shared ownership, in response to 
the decrease in the load condition. 

17. The system of claim 1, further comprising: means for 
updating the forwarding frequency, in response to a for- 
warded request. 

18. The system of claim 1, further comprising: 
multicasting means for multicasting a shift request mes- 
sage to one or more of the other cooperating cache 
servers so that subsequent forward requests will be 
shifted. 

19. The system of claim 1, wherein said shifting means for 
shifting one or more of said forwarded requests comprises 
means for communicating a copy of the object from an 
owning cache server to one or more of said cooperating 
cache servers. 

20. The system of claim 1, further comprising, 

means for calculating the load condition of each cache 

server in past intervals; 
means for computing a mean load of all cache servers in 

past intervals; and 
means for finding the cache servers that exceed a thresh- 60 

old above said mean load. 

21. The system of claim 1, wherein the load condition of 
said cooperating cache server comprises a weighted sum of 
a count of said forwarded requests, and a count of direct 
requests to said cooperating cache server. 65 

22. The system of claim 1, wherein said object is disposed 
at an object level and wherein each object may further be 



one of a plurality of objects in a partition of objects, said 
partition having a partition level further comprising means 
for maintaining cache information at one or more of: the 
object level for each object; and the partition level of a 
partition of objects. 

23. The system of claim 22, wherein said cache informa- 
tion of said object level or said partition level, comprises the 
forwarding frequency associated with the object. 

24. The system of claim 1, further comprising: 
centralized logical load monitor means for maintaining 

the forwarding frequency and the load condition for 
said cooperating cache servers. 

25. The system of claim 1, wherein said cooperating cache 
servers comprise cooperating proxy cache servers. 

26. The system of claim 21, further comprising: 

a logical directory server means for locating objects and 

forwarding requests for locally missed objects; 
means for maintaining a caching table and a load table, 

coupled to said directory server; 
cache server means for interrogating said directory server 

for object locations in other cache servers for a locally 

missed object; and 

directory server means for load balancing requests among 
said cache servers by manipulating said caching table, 
in response to requests for object locations. 

27. The system of claim 1, further comprising: 
multicasting means on each cache server for multicasting 

to a list of cooperating cache servers to locate a copy of 
a locally missed object; 
said shifting means comprising means for excluding over- 
loaded cache servers from a subset of neighboring 
cache servers for multicasting. 

28. A cache server load balancing method, comprising the 
steps of: 

receiving forwarded requests which have been forwarded 
from one of a plurality of cooperating cache servers in 
response to a cache miss for an object on the cooper- 
ating cache server; and 

shifting one or more of said forwarded requests for the 
object between cooperating cache servers based on 
dynamically maintained server load conditions and 
forwarding frequency, said forwarding frequency com- 
prising the number of times requests for the object have 
been forwarded. 

29. The method of claim 28, said shifting step further 
comprising the steps of: 

periodically monitoring the load condition on and the 
forwarding frequency to an owning cache server; and 

proactive ly shifting one or more subsequent forwarded 
requests for the cached object from the owning cache 
server to one or more of said cooperating cache servers, 
in response to said monitoring. 

30. The method of claim 28, said shifting step further 
comprising the step of checking the load condition and 
forwarding frequency, in response to the forwarded request. 

31. The method of claim 28, wherein said shifting com- 
prises the step of modifying an ownership for the object to 
a shared ownership between two or more of said cooperating 
cache servers. 

32. The method of claim 31, further comprising the step 
of merging said shared ownership in response to change in 
the load condition. 

33. Thu method of claim 28, further comprising the step 
of locally monitoring the load on each cooperating cache 
server. 
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34. The method of claim 33, further comprising the step communicating a copy of the object from an owning 
of: cache server to one or more of said cooperating cache 

a distributed load monitor monitoring and maintaining a servers; and 

local load condition, the forwarding frequency and said cooperating cache server receiving and caching the 

ownership information for cached objects on said each 5 copy of the object. 

cooperating cache server. 46. The method of claim 28, further comprising the steps 

35. The method of claim 34, further comprising the steps of, 

of: calculating the load condition of each cache server in past 

said cooperating cache servers periodically exchanging intervals; 

and maintaining one or more of: the load condition computing a mean load of all cache servere in past 

information; the forwarding frequency; and the own- intervals; and 

"*fu^?^r°^ • c -.X. • • .1. . finding the cache servers that exceed a threshold above 

36. The method of claim 34, further comprising the steps . f , , 

' r to r mean load. 

15 47. The method of claim 28, wherein the load condition 

said cooperating cache servers exchanging by piggyback- of said cooperating cache server can be a weighted sum of 

mg one or more of: the load condition information; the ^ ^^^t of said forwarded requests, and a count of direct 

forwarding frequency; and the ownership informaUon; requests to said cooperating cache server, 

with one or more of the forwarded requests and ^ y^e method of claim 28, wherein said object is 

responses. disposed at an object level and wherein each object may 

37. The method of claim 28, further comprising the step f^^ther be one of a plurality of objects in a partition of 
of: receiving a forwarded request and updating the forward- objects, said partition having a partition level further com- 

r ^ t • prising the step of maintaining cache information at one or 

38. The method of claim 34, further comprising the steps ^^^^ ^f. ^^e object level for each object; and the partition 

25 ^^^^^ ^ partition of objects, 

identifying a less loaded cooperating cache server; and 49. The method of claim 48, wherein said cache infor- 

communicating one or more of: a shift request; and a copy mation of said object level or said partition level comprises 

of the cached object, to said less loaded cooperating the forwarding frequency associated with the object. 

cache server. 50. The method of claim 49, further comprising the step 

39. The method of claim 38, further comprising the steps 30 of: 

a distributed load monitor monitoring and locally main- 
said less loaded cooperating cache server receiving said taining load conditions, forwarding frequency and 

shift request; and ownership information for cached objects on each 

said less loaded cooperating cache server requesting a cache server. 

copy of the object from an originating object server, in 35 51. The method of claim 50, further comprising the steps 

response to said shift request. of: 

40. The method of claim 38, wherein the copy is obtained said cooperating cache servers periodically exchanging 
via one or more of an intranet, WAN or Internet. one or more of the load condition, the forwarding 

41. The method of claim 28, further comprising the step frequency and the ownership information. 

of multicasting a shift request message to one or more of the 40 52. The method of claim 49, further comprising the steps 

other cooperating cache servers so that subsequent forward of; 

requests will be shifted. g^jj cooperating cache servers exchanging by piggyback- 

42. The method of claim 41, further comprising the step ^^ore of: the load condition; the forwarding 
*^f- frequency; and the ownership information; with one or 

the cooperating cache servers maintaining one of a local ^^^^ forwarded requests and responses, 

copy of a caching table and modifying a hash function; 53. in a collection of cooperating cache servers, where 

and each cache server can receive direct requests and forwarded 

the cooperating cache servers modifying the ownership requests, and upon a cache miss, a request can be forwarded 

information by one of: updating a local copy of a to an owning cache server caching said object, a load 

caching table; and modifying a hash function. balancing method comprising the steps of, 

43. The method of claim 42, further comprising the steps monitoring a dynamically maintained load condition and 

a forwarding frequency for said cooperating cache 

modifying the ownership for the object to a shared servers, said forwarding frequency comprising the 

ownership between at least two of said cooperating number of times requests for the object have been 

cache servers; and forwarded; and 

said cooperating cache servers forwarding subsequent shifting one or more forwarded requests, which have 

object requests to one or more less loaded shared already been forwarded, from one cooperating cache 

owners of the object. server to a second cooperating cache server based on a 

44. The method of claim 43, further comprising the steps change in the load condition and the forwarding fre- 
of: quency. 

delecting a decrease in the load condition for a shared 54. The method of claim 53, wherein said step of moni- 

object; and toring the load condition comprises the steps of, 

merging the shared ownership, in response to the decrease calculating the load condition of each cache server in past 

in the load condition. 65 intervals; 

45. 'llie method of claim 28, wherein said shifting one or computing a mean load of all cache servers in past 
more of said forwarded requests comprises the steps of: intervals; and 
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finding those cache servers that exceed a threshold above 
said mean load. 

55. The method of claim 53, wherein said shifting step can 
be performed in response to one or more of: said forwarded 
requests from said cooperating cache servers; and periodi- 
cally monitoring the load condition and the forwarding 
frequency. 

56. The method of claim 53, further comprising the step 
of a centralized logical load monitor maintaining the for- 
warding frequency and the load condition for the cooperat- 
ing cache servers. 

57. The method of claim 53, wherein the load condition 
of said cache server can be a weighted sum of: a count of 
forwarded requests; and a count of direct requests to said 
cache server. 

58. The method of claim 53, further comprising the step 
of maintaining cache information at each object level or at 
a partition of objects level. 

59. The method of claim 58, wherein said cache infor- 
mation of the object level or the partition level comprises the 
forwarding frequency of requests through said load monitor 
to said object. 

60. The method of claim 53, wherein said cooperating 
cache servers comprise cooperating proxy cache servers. 

61. The method of claim 53, further comprising the steps 

of: 

a logical directory server maintaining a caching table and 
a load table; 

said cache servers interrogating said directory server for 
object locations in other cache servers for a locally 
missed object; and 

said directory server load balancing requests among said 
cache servers by manipulating said caching table, in 
response to requests for object locations. 

62. The method of claim 56, fiirther comprising the steps 

of: 

each cache server multicasting to a list of cooperating 
cache servers to locale a copy of a locally missed 
object; and 

said shifting step comprising the step of excluding over- 
loaded cache servers from a subset of neighboring 
cache servers for multicasting. 

63. A program storage device readable by a machine, 
tangibly executable a program of instructions executable by 
the machine to perform method steps for cache server load 
balancing, said method steps comprising: 

receiving forwarded requests which have been forwarded 
from one of a plurality of cooperating cache servers in 
response to a cache miss for an object on the cooper- 
ating, cache server; and 

shifting one or more of said forwarded requests for the 
object between cooperating cache servers based on 
dynamically maintained server load conditions and 
forwarding frequency, said forwarding frequency com- 
prising the number of times requests for the object have 
been forwarded. 

64. The program storage device of claim 63, said shifting 
step further comprising the steps of: 

periodically monitoring the load condition on and the 
forwarding frequency to an owning cache server; and 

proactively shifting one or more subsequent forwarded 
requests for the cached object from the owning cache 
server to one or more of said cooperating cache servers, 
in response to said monitoring. 

65. llie program storage device of claim 63, said shifting 
step further comprising the step of checking the load con- 
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dition and forwarding frequency, in response to the for- 
warded request. 

66. The program storage device of claim 63, wherein said 
shifting comprises the step of modifying an ownership for 

5 the object to a shared ownership between two or more of 
said cooperating cache servers. 

67. The program storage device of claim 66, further 
comprising the step of merging said shared ownership in 
response to change in the load condition. 

68. The program storage device of claim 63, further 
comprising the step of locally monitoring the load on each 
cooperating cache server. 

69. The program storage device of claim 68, further 
comprising the step of: 

a distributed load monitor monitoring and maintaining a 
local load condition, the forwarding frequency and 
ownership information for cached objects on said each 
cooperating cache server. 

70. The program storage device of claim 69, further 
comprising the steps of: 

20 said cooperating cache servers periodically exchanging 
and maintaining one or more of: the load condition 
information; the forwarding frequency; and the own- 
ership information. 

71. The program storage device of claim 69, further 
25 comprising the steps of: 

said cooperating cache servers exchanging by piggyback- 
ing one or more of: the load condition information; the 
forwarding frequency; and the ownership information; 
with one or more of the forwarded requests and 
responses. 

72. The program storage device of claim 69, further 
comprising the steps of: 

identifying a less loaded cooperating cache server; and 
communicating one or more of: a shift request; and a copy 

of the cached object, to said less loaded cooperating 

cache server. 

73. The program storage device of claim 72, further 
comprising the steps of: 

said less loaded cooperating cache server receiving said 

shift request; and 
said less loaded cooperating cache server requesting a 

copy of the object from an originating object server, in 

response to said shift request. 

74. liie program storage device of claim 72, wherein the 
copy is obtained via one or more of an intranet, WAN or 
Internet. 

75. The program storage device of claim 63, further 
comprising the step of: receiving a forwarded request and 
updating the forwarding frequency. 

76. The program storage device of claim 75, further 
comprising the step of: 

multicasting a shift request message to one or more of the 
other cooperating cache servers so that subsequent 
55 forward requests will be shifted. 

77. The program storage device of claim 76, further 
comprising the step of: 

the cooperating cache servers maintaining one of: a local 
copy of a caching table; and a hash function; and 
60 the cooperating cache servers modifying the ownership 
information by one of: updating the local copy of a 
caching table; and modifying a hash function. 

78. The program storage device of claim 77, further 
comprising the steps of: 

65 modifying the ownership for the object to a shared 
ownership between at least two of said cooperating 
cache servers; and 
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said cooperating cache servers forwarding subsequent to a cooperating cache server caching said object, said 

object requests to one or more less loaded shared method steps comprising: 

owners of the object. monitoring a load condition and a forwarding frequency 

79. The program storage device of claim 78, further cooperating cache servers; and 
comprising the steps of: 5 r . j r 

, . , f ' I- . J shiftmg one or more forwarded requests from one coop- 

detectmg a decrease m the load condition tor a shared *• u * j *- u 

. , eratmg cache server to a second cooperating cache 

° ^ ' server based on a change in the load condition and the 

merging the shared ownership, in response to the decrease forwarding frequency. 

on'"J^^ . . r . . u . .. 10 89. The program storage device of claim 88, wherein said 

80. The program storage device of claim 63, wherem said monitoring the load condition comprises the steps of, 
shifting one or more of said forwarded requests comprises i & ft 

the steps of- calculating the load condition of each cache server in past 

communicating a copy of the object from an owning intervals, 

cache server to one or more of said cooperating cache computing a mean load of all proxy cache servers in past 

servers; and intervals; and 

said cooperating cache server receiving and caching the finding those proxy cache servers that exceed a threshold 

copy of the object. above said mean load. 

81. The program storage device of claim 63, further 90. The program storage device of claim 88, wherein said 
comprising the steps of, 20 shifting step can be performed in response to one or more of: 

calculating the load condition of each cache server in past said forwarded requests from said cooperating cache serv- 

intervals; ers; and 

computing a mean load of all cache servers in past periodically monitoring the load condition and the for- 

intervals; and warding frequency. 

finding the cache servers that exceed a threshold above P,^°g^™ ^^^'^^^ device of claim 88, further 

,r„ t , comprising the step 01 a centrauzed logical load momtor 

said mean load. • ■ , ^ r j . 1 j i- • 

82. The program storage device of claim 63, wherein the ma.ntainmg the forwarding frequency and the load condition 
load condition of said cooperating cache server can be a for the cooperating cache servers. 

weighted sum of a count of said forwarded requests, and a "2. ITie program storage device of claun 91, further 
count of direct requests to said cooperating cache server. "'^ compnsmg e s eps o . 

83. The program storage device claim 63, wherein said each cache server multicasting to a fist of cooperating 
object is disposed at an object level and wherein each object cache servers to locate a copy of a locally missed 
may further be one of a plurality of objects in a partition of object; and 

objects, said partition having a partition level further com- said shifting step comprising the step of excluding over- 
prising the step of maintaining cache information at one or loaded cache servers from a subset of neighboring 
more of: the object level for each object; and the partition cache servers for multicasting. 

level of a partition of objects. 93. The program storage device of claim 88, wherein the 

84. The program storage device of claim 83, wherein said load condition of said cache server can be a weighted sum 
cache information of said object level or said partition level, of: a count of forwarded requests; and a count of direct 
comprises the forwarding frequency associated with the requests to said cache server. 

object. 94. The program storage device of claim 88, wherein said 

85. Hie program storage device of claim 84, further object is disposed at an object level and wherein each object 
comprising the step of: may further be one of a plurality of objects in a partition of 

a distributed load monitor monitoring and locally main- objects, said partition having a partition level further com- 

taining load conditions, forwarding frequency and prising the" step of maintaining cache information at the 

ownership information for cached objects on each object level for each object; and the partition level of a 

cache server. partition of objects. 

86. The program storage device of claim 85, further 95. The program storage device of claim 88, wherein said 
comprising the steps of: cache information of the object level or the partition level 

said cooperating cache servers periodicaUy exchanging '° comprises the forwarding frequency of requests through said 

one or more of the load condition, the forwarding 'oad niomtor to said object , , . ^ . 

frequency and the ownership information. ^« P™S'7 ^'""^ ^""l 

87. The program storage device of claim 84, further cooperating cache servers comprise cooperating proxy 
• . r. * c cache servers. 

comprising the steps of: 55 ^ , , . f 1 • 00 ^ 

, . 97. The program storage device of claim 88, further 

said cooperatmg cache servers exchangmg by piggyback- comprising the steps of: 

ing one or more of: the load condition; the forwarding ... , . , , , , , t , 

frequency; and the ownership information; with one or mamtaimng a caching table and a load table; 

more of the forwarded requests and responses. receiving requests for object locations in other cache 

88. A program storage device readable by a machine, 60 servers for a locally missed object; and 

tangibly embodying a program of instructions executable by load balancing requests among said cache servers by 

the machine to perform method steps for cache server load manipulating said caching table, in response to said 

balancing in a collection of cooperating cache servers, where requests for object locations, 
each cache server can receive direct requests and forwarded 

requests, and upon a cache miss, a request can be forwarded * * * * ♦ 
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